跳转至

GS-VLA：基于高斯溅射的冻结 VLA 策略即插即用视点归一化

文章背景与核心概要

当前视觉-语言-动作（VLA）模型在机器人操控任务中表现出色，但其性能高度依赖于训练与部署时相机视点的一致性。哪怕是微小的相机位置偏移，也会导致任务成功率发生灾难性下降。传统的解决方案（如大规模微调或生成式数据增强）不仅计算成本高昂，还容易引发灾难性遗忘。

为了解决这一痛点，本文提出了 GS-VLA 框架。该研究的核心洞察在于：在局部性假设下（即相机扰动相对于工作空间保持在一个较小的有界区域内），视点偏移可以转化为一个局部的新视点合成问题，进而简化为与场景和策略无关的去遮挡任务。

GS-VLA 创新性地在冻结的 VLA 策略前端直接拼接了一个仅有 400 万参数的 3D 高斯归一化器（3D-Gaussian canonicalizer）。实验结果表明，在无需修改任何策略权重的情况下，GS-VLA 在不同的策略架构、未见任务集以及多种扰动尺度下，均成功恢复了因视点变化而丢失的大部分性能。

📌 Summary

GS-VLA is a lightweight, plug-and-play framework designed to improve the robustness of Vision-Language-Action (VLA) policies against viewpoint shifts without requiring policy retraining.

  • The Problem: Current VLA performance assumes training and deployment camera setups are identical. Minor camera mount displacements can catastrophically degrade task success rates (e.g., dropping from ~90% to ~10% on the LIBERO benchmark). Existing solutions like large-scale fine-tuning or generative data augmentation are computationally expensive and risk catastrophic forgetting.
  • The Core Insight: Viewpoint shifts can be framed as a localized novel-view synthesis problem. Under the Locality assumption (camera perturbations remain within a small bounded region relative to the workspace), viewpoint normalization simplifies to a scene- and policy-independent disocclusion task.
  • The Solution: GS-VLA prepends a compact 4-million-parameter 3D-Gaussian canonicalizer directly to a frozen VLA policy.
  • Key Results: Without modifying policy weights, GS-VLA successfully recovers a large portion of lost performance across three distinct dimensions:
  • Policy architectures
  • Unseen task suites
  • Various perturbation scales